3 research outputs found
Deep Active Learning in the Presence of Label Noise: A Survey
Deep active learning has emerged as a powerful tool for training deep
learning models within a predefined labeling budget. These models have achieved
performances comparable to those trained in an offline setting. However, deep
active learning faces substantial issues when dealing with classification
datasets containing noisy labels. In this literature review, we discuss the
current state of deep active learning in the presence of label noise,
highlighting unique approaches, their strengths, and weaknesses. With the
recent success of vision transformers in image classification tasks, we provide
a brief overview and consider how the transformer layers and attention
mechanisms can be used to enhance diversity, importance, and uncertainty-based
selection in queries sent to an oracle for labeling. We further propose
exploring contrastive learning methods to derive good image representations
that can aid in selecting high-value samples for labeling in an active learning
setting. We also highlight the need for creating unified benchmarks and
standardized datasets for deep active learning in the presence of label noise
for image classification to promote the reproducibility of research. The review
concludes by suggesting avenues for future research in this area.Comment: 20 pages, PhD literature revie
Comparision Of Adversarial And Non-Adversarial LSTM Music Generative Models
Algorithmic music composition is a way of composing musical pieces with
minimal to no human intervention. While recurrent neural networks are
traditionally applied to many sequence-to-sequence prediction tasks, including
successful implementations of music composition, their standard supervised
learning approach based on input-to-output mapping leads to a lack of note
variety. These models can therefore be seen as potentially unsuitable for tasks
such as music generation. Generative adversarial networks learn the generative
distribution of data and lead to varied samples. This work implements and
compares adversarial and non-adversarial training of recurrent neural network
music composers on MIDI data. The resulting music samples are evaluated by
human listeners, their preferences recorded. The evaluation indicates that
adversarial training produces more aesthetically pleasing music.Comment: Submitted to a 2023 conference, 20 pages, 13 figure
PuoBERTa: Training and evaluation of a curated language model for Setswana
Natural language processing (NLP) has made significant progress for
well-resourced languages such as English but lagged behind for low-resource
languages like Setswana. This paper addresses this gap by presenting PuoBERTa,
a customised masked language model trained specifically for Setswana. We cover
how we collected, curated, and prepared diverse monolingual texts to generate a
high-quality corpus for PuoBERTa's training. Building upon previous efforts in
creating monolingual resources for Setswana, we evaluated PuoBERTa across
several NLP tasks, including part-of-speech (POS) tagging, named entity
recognition (NER), and news categorisation. Additionally, we introduced a new
Setswana news categorisation dataset and provided the initial benchmarks using
PuoBERTa. Our work demonstrates the efficacy of PuoBERTa in fostering NLP
capabilities for understudied languages like Setswana and paves the way for
future research directions.Comment: Accepted for SACAIR 202